Pointing device

ABSTRACT

A pointing device to operate a portable electronic device using a finger is disclosed. The pointing device comprises a light emitting means for illuminating a subject; a hole through which light from the light emitting means is transmitted; an image-acquisition area for taking an image of the subject from the transmitted light; an image-formation means for forming an image by focusing the light reflected from the image-acquisition area; a conversion means for converting the image formed by the image-formation means into an electric signal; and an operation means for detecting the change of the image and calculating the amount of the change using the electric signal output from the conversion means. The subject is preferably the surface of a finger, a lattice, or any perceivable pattern. The pointing device can be embodied on portable electronic devices such as cellular phones and PDAs because it is small in size.

TECHNICAL FIELD

The present invention relates to a pointing device and, moreparticularly, to a pointing device to operate a handheld terminal usinga finger.

BACKGROUND ART

Pointing devices generally include XY tablets, trackballs and mouse thatare used for stationary devices such as desktop computers, and touchscreen panels (hereinafter referred to as “TSPs”) and touch pads thatare used for portable electronic devices such as notebook computers.

The XY tablet is a means for calculating position variation of themagnetic field, which is generated by the flow of electric current, witha separate magnetic field sensor. In detail, electric currents aresuccessively applied to a lattice consisting of two-dimensionalconductors toward X or Y direction to generate a magnetic field. The XYtablet comprises a magnetic field sensor connected with an XY tabletbody.

The trackball includes an appropriate fixing member to prevent a rollingspherical ball from getting derailed, and at least two rotation speeddetectors that detect the rotation of the ball. The trackball representsthe rotation of the ball as two-dimensional movement using the rotationspeed detectors.

The ball mouse is a device applying the principle of the trackball bycontraries. In the ball mouse, instead of a ball, the trackball deviceitself moves to rotate the ball relatively. Therefore, the ball mouserepresents the two-dimensional movement using the movement of thetrackball device itself.

The TSP includes two flat resistance films positioned adjacently eachother. If a user presses the panel with a sharp-end means such as aball-point-pen, the resistance films in the pressed point contact eachother to form a resistance circuit. Through an appropriate combinationof these circuits, the position of contact between the resistance filmscan be calculated two-dimensionally.

FIG. 1 is a schematic diagram illustrating a conventional optical mousedevice. As shown in FIG. 1, the optical mouse device has a two-dimensionoptical sensor array (10) that comprises CMOS image sensors (hereinafterreferred to as “CIS”). In addition, an optical means (15) such as a lensis added to the two-dimension optical sensor array (10) to perceiverelative coordinates of the mouse position through reflective light froma surface (20). There is no change of the coordinates if the opticalmouse is not moved. If the optical mouse moves, the coordinates of theoptical mouse change.

Such an optical mouse device calculates the movement distance of theoptical mouse using an appropriate operation means (25) and a motiondetector (30) based on the coordinate change. Particularly, in order todetermine the movement distance, the motion detector (30) adopts amotion estimation method. The conventional optical mouse sends lightthrough a hole (40) formed through the bottom of its casing (35) towardthe surface (20) of an object on which the optical mouse is positioned.The light emitting means is generally a light emitting diode (LED). Inaddition, the conventional optical mouse device adopts a structure tominimize errors caused by specular light directly reflected from thesurface (20).

However, these conventional pointing devices have several disadvantages.For example, the TSP can activate a desired icon by pressing the iconwith a sharp-end tool to operate a machine. However, the TSP requiresusers to use both hands, holding the TSP by one hand and using thesharp-end tool by the other hand. In addition, users are disabled to usethe TSP whenever the sharp-end tool is mislocated or not available. Theball mouse and optical mouse are difficult to be applied to portableelectronic devices of small size because of the intrinsic limitations oftheir movement structure.

FIG. 2 shows an example of a portable electronic device having aconventional pointing device. Although FIG. 2 shows a notebook with apointing device, all portable electronic devices including PDA (personaldigital assistance) as well as the notebook can employ the pointingdevice. As shown in FIG. 2, the notebook (50) has a flat-panel displayand GUI (graphic user interface) to maximize user's convenience. Usersneed to use a pointing device such as a touch pad (70), a TSP or a mouseto utilize GUI of the portable electronic device.

However, in the conventional portable electronic devices, a considerablepart in the top area has to be assigned to the touch pad. As analternative, the mouse device can be coupled to a connection port of theportable electronic device, but, in this case, the user has to carryseparately the mouse with the portable electronic device.

Korean Patent Publication No. 2002-14430 discloses a portable wirelessinformation terminal having a pointing device to effectively utilizeapplication programs under the GUI environment. In the above-mentionedportable wireless information terminal, the pointing device, whichincludes a ball and a sensor for sensing rotational position, is mountedon the lower side of the information terminal to freely move a cursor onan LCD (liquid crystal display).

However, the above-mentioned prior art has a problem that it isrestricted to a ball mouse because the pointing device employs a ballinstalled in a concave groove of a housing. In addition, the ball isprotruded outside from the lower side of the portable terminal, therebycausing inconvenience in use.

DISCLOSURE OF INVENTION

The present invention provides a pointing device comprising a lightemitting means for illuminating a subject; a hole through which lightfrom the light emitting means is transmitted; an image-acquisition areafor taking an image of the subject from the transmitted light; animage-formation means for forming an image by focusing the lightreflected from the image-acquisition area; a conversion means forconverting the image formed by the image-formation means into anelectric signal; and an operation means for detecting the change of theimage and calculating the amount of the change using the electric signaloutput from the conversion means. Here, the subject is preferably thesurface of a finger, a lattice, or any perceivable pattern.

In a second embodiment, the present invention provides a pointing devicecomprising a light emitting means; a light guide structure for guidinglight from the light emitting means to a subject; an image-acquisitionarea for taking an image of the subject from the guided light; animage-formation means for forming an image by focusing the lightreflected from the image-acquisition area; a conversion means forconverting the image formed by the image-formation means into anelectric signal; and an operation means for detecting the change of theimage and calculating the amount of the change using the electric signaloutput from the conversion means. Here, the subject is preferably thesurface of a finger, a lattice, or any perceivable pattern.

As a third embodiment, the present invention provides a pointing devicecomprising a light emitting means; a light guide structure for guidinglight from the light emitting means to a subject; an image-acquisitionarea for taking an image of the subject from the guided light; animage-formation means for forming an image by focusing the lightreflected from the image-acquisition area; a housing coupled to theimage-formation means; a conversion means for converting the imageformed by the image-formation means into an electric signal; a printedcircuit board on which the conversion means is fixed; a cover forprotecting the light emitting means, the image-formation means, thehousing, the conversion means, and the printed circuit board; and anoperation means for detecting the change of the image and calculatingthe amount of the change using the electric signal output from theconversion means.

BRIEF DESCRIPTION OF THE DRAWINGS

Further objects and advantages of the invention can be more fullyunderstood from the following detailed description taken in conjunctionwith the accompanying drawings, in which:

FIG. 1 is a schematic diagram illustrating a conventional optical mousedevice;

FIG. 2 shows an example of a portable electronic device having aconventional pointing device;

FIG. 3 is a schematic diagram illustrating a pointing device accordingto an embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating a pointing device accordingto another embodiment of the present invention;

FIG. 5 is a schematic diagram illustrating a pointing device accordingto a third embodiment of the present invention;

FIG. 6 shows examples of subjects available for a pointing deviceaccording to the present invention;

FIG. 7 is a diagram illustrating how the pointing device of the presentinvention works;

FIG. 8 shows an example of portable electronic device mounted with apointing device according to the present invention;

FIG. 9 is a schematic diagram illustrating a pointing device with acontact sensor according to the present invention;

FIG. 10 is a schematic diagram illustrating a two-dimensional pointingdevice with an integral structure;

FIG. 11 is a schematic diagram illustrating a pointing device accordingto another embodiment of the present invention; and

FIG. 12 shows an example of portable electronic device equipped with apointing device according to the present invention.

REFERENCE

100, 440: light emitting means 110: contact subject 120, 400:image-formation means 130, 150: conversion means 140: motion detector150: operation means 160: casing 170: hole 180: transparent plate 190,405: light guide structure 300: portable electronic device 310: pointingdevice 320: mark 330: selection button 350: contact sensor 360: controlpart 410: housing 420: printed circuit board 430: cover 500: pointingdevice module 510: contact area 520: switch 530: screen 540: pointer

BEST MODE FOR CARRYING OUT THE INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings.

FIG. 3 is a schematic diagram of a pointing device in accordance with anembodiment of the present invention. Referring to FIG. 3, the pointingdevice of the present invention calculates the distance and directionfor a pointer to be moved by acquiring and analyzing an image formed bylight that is transmitted from a light emitting means (100) to a contactsubject (110). This method presents a similar effect to that of anoptical mouse, which moves on the fixed surface of a desk or a flatpart. Here, the pointer means an indicator on a screen of displaydevice, which is moved by a pointing device such as a mouse.

Such a pointing device is mounted on a portable electronic device andacquires changing images. In other words, a user can convenientlycontrol a pointer on the screen of portable electronic device using thesurface of his/her finger. The movement of a finger causes the change ofimage in the image-acquisition area. The distance and direction for thepointer to be moved can be calculated through the analysis of the changeof image due to the movement of a finger.

The pointing device according to the present invention comprises a lightemitting means (100), an image-acquisition area (not shown), animage-formation means (120), a conversion means (130), a motion detector(140), and an operation means (150). These component parts are describedin detail.

The light emitting means (100) emits light to illuminate a contactsubject (110) controlling a pointer. The contact subject (110) ispreferably the surface of a user's finger, a lattice, or any perceivablepattern. The light emitting means (100) is preferably an LED, a laserdiode, or an organic electroluminescence.

The image-acquisition area (not shown) acquires movement data using thelight illuminated from the light emitting means (100). Theimage-acquisition area is positioned at a predetermined distance fromthe image-formation means (120). The image-acquisition area ispreferably a housing with a transparent member whose surface in contactwith the contact subject (110) is flat and made of transparent material.The housing is preferably coated to prevent the surface of theimage-acquisition area from damage or contamination. Additionally, theimage-acquisition area may be a virtual plane positioned at apredetermined distance from the image-formation means (120).

The image-formation means (120) forms an image on the opposite side byfocusing the light reflected from the image-acquisition area. Theimage-formation means (120) may be an optical lens, preferably aspherical or non-spherical lens or a mirror.

The conversion means (130) detects the analog image formed by theimage-formation means (120) and converts it into a digital image. Theconversion means (130) is preferably an optical sensor array in which aplurality of CMOS image sensors or CCD (charge coupled device) imagesensors are arranged in two-dimensional form.

The motion detector (140) perceives the extent of movement through thedigital image received from the conversion means (130) using a motionestimation method. The motion detector (140) uses a motion estimator.

The operation means (150) receives the extent of movement, e.g., shiftdata from the motion detector (140) and calculates the distance anddirection for the pointer to be moved. The operation means (150) iscoupled to the pointing device or the processor of a machine on whichthe pointing device is mounted. Accordingly, the processor can controlthe pointer on the screen of a display device so that it can be freelymoved toward a desired direction by a desired distance.

In FIG. 3, the reference number 160 is a casing of a pointing device ora casing of a portable electronic device equipped with the pointingdevice. The reference number 170 is a hole formed through the casing(160). The light from the light emitting means (100) is reflected fromthe finger surface through the hole (170).

For example, the light from a light emitting diode is illuminated ontothe finger surface and reflected according to a pattern of the fingersurface. The light reflected from the finger surface forms an image onthe surface of the CMOS image sensor or CCD image sensor array throughthe lens. The formed image is converted into an electric signal by theCMOS image sensor or the CCD image sensor array and entered into asignal processing part to be changed into a digital image.

The above-mentioned image acquisition is performed very rapidly on areal-time base. The motion estimator detects the extent of changebetween images by comparing the images formed in two adjacent timesequences. The detected movement implies movement of a finger in theadjacent time sequences. Thus, the present invention can embody thepointing device such as a mouse device of a computer using the fingermovement.

A conventional mouse device needs a large flat area on which the mousedevice is moved, but the present invention can embody the pointingdevice within a small space by minimizing the size of theimage-acquisition area.

FIGS. 4 and 5 are schematic diagrams illustrating a pointing deviceaccording to embodiments of the present invention. The pointing deviceillustrated in FIG. 4 additionally comprises a transparent plate (180)and another LED (104) compared to the pointing device in FIG. 3.

In general, the change of distance between the finger surface and theimage-acquisition area causes images to be unclear, thereby making themovement analysis difficult. However, by using the transparent plate(180) to shorten the focal length of lens, the pointing device of thepresent invention can avoid this problem. In detail, in order tomaintain uniformly the distance between the contact subject and theimage-acquisition area, the present invention employs the transparentplate (180) so that the finger moves on the flat area.

In addition, when the finger surface (110) moves, the dimension andposition of the shadow of the finger may change to cause error becauseof an outside three-dimension lighting. This problem can be solved withtwo LEDs (102, 104).

Referring to FIG. 5, the pointing device additionally comprises a lightguide structure (190). The light guide structure (190) transforms thelight transmitted to the transparent plate (180) into the light similarto planar light, thereby minimizing error occurrence due to change oflighting conditions and distributing uniformly the light from the lightemitting means (100) to constantly maintain desired brightness in thepredetermined area. In addition, the factors related to the light guidestructure (190) are adjusted appropriately in order to minimizetransmission loss of light or image.

FIG. 6 shows examples of subjects available for a pointing deviceaccording to the present invention. In FIG. 6, (a) is the fingerprint ofa finger, (b) is a lattice, and (c) is an example of any perceivablepattern. The pointing device in accordance with the present inventionpreferably uses the surface of a finger, a lattice or a perceivablepattern as the contact subject.

FIG. 7 is a diagram illustrating how the pointing device of the presentinvention works. In FIG. 7 (a), area A (200) is an image formed on aCMOS image sensor or a CCD image sensor at the initial time T=0 and areaB (210) is an image after the area A (200) moves 4 pixels to the rightand 3 pixels downward during a predetermined time T1. In FIG. 7 (b),area C (220) is an image formed on a CMOS image sensor or a CCD imagesensor at the initial time T=O and area D (230) is an image after thearea C (220) moves 2 pixels to the left and 3 pixels upward during apredetermined time T2. Thus, the pointing device in accordance with thepresent invention can recognize the direction and distance of a movementmade.

FIG. 8 shows a portable electronic device equipped with a pointingdevice according to the present invention. As shown in FIG. 8, theportable electronic device (300) includes a PDA. The PDA is equippedwith the above-mentioned pointing device (310). The pointing device(310) is not exposed outside and a cross-shaped mark (320) is printed onthe location of the pointing device (310) to represent the place onwhich a finger, for example, is positioned to control a pointer.

Preferably, the portable electronic device (300) additionally comprisesat least a selection button (330) connected to the pointing device(310). The selection button (330) is used to select a target with thepointer moved by the pointing device (310) or to enter a command. Here,the portable electronic device (300) uses a secondary battery as a mainpower source.

FIG. 9 is a schematic diagram illustrating a pointing device with acontact sensor according to the present invention. Preferably, the lightemitting means in a portable electronic device has to be operated onlyduring the use of the pointing device because a considerable electricpower is required to operate the light emitting means. Therefore, as ameans to determine if the pointing device is in use, a contact sensor(350) is attached around the image-acquisition area. When a fingertouches the contact sensor (350) or is placed near the contact sensor(350), the control part (360) operates the light emitting means so thatthe pointing device begins to work. The contact sensor (350) maypreferably perform the role of selection switch, which is similar todouble-click of a conventional mouse device, for selecting the presentposition or a predetermined function indicated by a pointer, based on apredetermined time interval for which the finger touches. For the roleof the selection switch, “contact”, “non-contact”, and “contact have tobe performed in sequence in a short time interval.

The contact sensor (350) may be internally connected with on-offterminals of the conversion means and the light emitting means so as tocontrol the on or the off state of the conversion means and the lightemitting means according to the “contact” or “non-contact” status withthe subject using hardware. The output of the contact sensor (350) maybe connected with the control part (210) so as to control the on or theoff state of the conversion means and the light emitting means usingsoftware.

In addition, the contact sensor (350) is preferably placed around theimage acquisition area within a radius of about 3 cm from the center ofthe image acquisition area and can be embodied by both contact andnon-contact fashion.

When a user employs a portable electronic device equipped with thepointing device, he or she may use a program that does not require thepointing device. In that case, he or she may push the contact sensorinstead of a desired button by mistake. To avoid the accidentaloperation of the pointing device due to such a mistake, the portableelectronic device with the pointing device may be programmed so that thecontact sensor operates the pointing device in case of a programrequiring the pointing device and the pointing device is turned off incase of a program requiring no pointing device.

For example, where a game using only arrow keys does not need thepointing device, the pointing device is automatically turned off bymeans of an appropriate programming while the game is displayed on ascreen.

FIG. 10 is a diagram illustrating a two-dimensional pointing device withan integral structure. As shown in FIG. 10, the pointing device inaccordance with the present invention is preferably used for small-sizedportable electronic devices such as cellular phones and, therefore, theminiaturization of the pointing device is essential. It is desirable tounite the image-formation means (400) with the fixing frame (410).Preferably, it is desirable to unite the light guide structure (405),the image-formation means (400), and the housing (410) as an integralstructure. The spherical or non-spherical lens is united into theintegral structure as a part. Such an integral structure improvesoperation efficiency and productivity by minimizing occurrence ofdefectiveness due to tolerance in assembling.

A printed circuit board (420) is installed into the housing (410). Thelight emitting means (440), the conversion means (450), and partscontaining circuits for operation are mounted on the printed circuitboard (420). A cover (430) is used to maintain constantly the shape ofthe finger surface in contact with the area in the top of the pointingdevice. In addition, the cover (430) can protect the light emittingmeans (440), the image-formation means (400), the housing (410), theconversion means (450), and the printed circuit board (420) fromcontamination such as dust.

FIG. 11 is a schematic diagram illustrating a pointing device accordingto another embodiment of the present invention. FIG. 12 shows an exampleof a portable electronic device equipped with the pointing deviceaccording to the present invention.

As shown in FIGS. 11 and 12, the pointing device in a small-sizeportable electronic device such as a cellular phone is preferablyembodied by the selection method using pressure. There is a contact area(510) on the top of the united pointing device module (500) and a switch(520) under the pointing device module (500). If the contact area (510)is pressed, the pointing device module (500) capable of moving up anddown operates the switch (520). When the pointing device module (500) ispressed to operate the switch (520), a graphic icon indicated by apointer (540) on a screen (530) is selected and, then, the correspondingfunction is performed.

On the other hand, the pointing device of the present invention mayemploy the surface of a palm, the surface of the back of a hand, and alltypes of objects with a surface similar to a finger surface as well asthe finger surface as a means to control the pointer.

The foregoing embodiments are merely exemplary and are not to beconstrued as limiting the present invention. The present teachings canbe readily applied to other types of apparatuses. The description of thepresent invention is intended to be illustrative, and not to limit thescope of the claims. Many alternatives, modifications, and variationswill be apparent to those skilled in the art.

INDUSTRIAL APPLICABILITY

Thus, the pointing device in accordance with the present invention canbe embodied in a small space through the minimization of theimage-acquisition area. In addition, the pointing device needs not amouse pad or a flat surface. The pointing device can be embodied onportable electronic devices such as cellular phones and PDAs because itis small in size.

1. A pointing device comprising: a light emitting means for illuminatinga subject selected from the group consisting of a surface of a finger, alattice, and any perceivable pattern; a hole through which light fromthe light emitting means is transmitted; an image-acquisition area fortaking an image of the subject from the transmitted light; animage-formation means for forming an image by focusing the lightreflected from the image-acquisition area; a conversion means forconverting the image formed by the image-formation means into anelectric signal; and an operation means for detecting the change of theimage and calculating the amount of the change using the electric signaloutput from the conversion means.
 2. A pointing device comprising: alight emitting means; a light guide structure for guiding light from thelight emitting means to a subject selected from the group consisting ofa surface of a finger, a lattice, and any perceivable pattern; animage-acquisition area for taking an image of the subject from theguided light; an image-formation means for forming an image by focusingthe light reflected from the image-acquisition area; a conversion meansfor converting the image formed by the image-formation means into anelectric signal; and an operation means for detecting the change of theimage and calculating the amount of the change using the electric signaloutput from the conversion means.
 3. The pointing device as defined byclaim 1, wherein the light emitting means is selected from the groupconsisting of a light emitting diode, a laser diode, and an organicelectroluminescence.
 4. The pointing device as defined by claim 3,wherein the light emitting means comprises at least one light emittingdiode.
 5. The pointing device as defined by claim 1, wherein theconversion means is a CMOS image sensor or a CCD image sensor.
 6. Thepointing device as defined by claim 1, further comprising a selectionbutton for selecting a target with a pointer moved by the pointingdevice or entering a command.
 7. The pointing device as defined by claim1, wherein the image-formation means is one selected from the groupconsisting of a spherical or non-spherical lens and a spherical ornon-spherical mirror.
 8. A pointing device comprising: a light emittingmeans; a light guide structure for guiding light from the light emittingmeans to a subject; an image-acquisition area for taking an image of thesubject from the guided light; an image-formation means for forming animage by focusing the light reflected from the image-acquisition area; ahousing coupled to the image-formation means; a conversion means forconverting the image formed by the image-formation means into anelectric signal; a printed circuit board on which the conversion meansis fixed; a cover for protecting the light emitting means, theimage-formation means, the housing, the conversion means, and theprinted circuit board; and an operation means for detecting the changeof the image and calculating the amount of the change using the electricsignal output from the conversion means.
 9. The pointing device asdefined by claim 8, wherein the subject is one selected from the groupconsisting of a surface of a finger, a lattice, and any perceivablepattern.
 10. The pointing device as defined by claim 8, furthercomprising a contact sensor for determining whether the pointing deviceis in use.
 11. The pointing device as defined by claim 10, wherein thecontact sensor is embodied by means of a direct contact or non-contactfashion.
 12. The pointing device as defined by claim 10, wherein thecontact sensor controls an on-off state of the light emitting means orthe conversion means based on whether the contact sensor is touched by ahuman body or an object.
 13. The pointing device as defined by claim 10,wherein the contact sensor is positioned around the image-acquisitionarea within a radius of about 3 cm from the center of the imageacquisition area.
 14. The pointing device as defined by claim 10,wherein the contact sensor operates the pointing device only for aprogram requiring the pointing device.
 15. The pointing device asdefined by claim 10, wherein the contact sensor performs the role of aselection switch for selecting the present position on a predeterminedfunction indicated by a pointer.
 16. The pointing device as defined byclaim 15, wherein the role of the selection switch is performedaccording to the change of time interval between contact and non-contactto the contact sensor.
 17. The pointing device as defined by claim 8,wherein the light guide structure, the image-formation means, and thehousing are united as an integral structure.
 18. The pointing device asdefined by claim 1, wherein the image-acquisition area is coated inorder to prevent damage or contamination of the image-acquisition area.19. The pointing device as defined by claim 1, wherein the operationmeans receives the electric signal from the conversation means anddetermines the distance and direction for a pointer to be moved bycalculating the electric signal.
 20. The pointing device as defined byclaim 1, wherein the image-acquisition area is a transparent member or avirtual plane positioned at a predetermined distance from theimage-formation means.